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ABSTRACT 

Existence,  strict  stationary  and  ergodicity  of  Bilinear  Time 
Series  Models  for  a  given  input  White  Noise  process  and  parameter 
values  is  studied  in  detail  in  this  paper.  Using  ergodicity  of  the 
model,  estimation  of  the  parameters  by  the  method  of  moments  is 
suggested  and  some  comparisons  are  made  with  the  method  of  least 
squares ,  x' 

Key  Words;  Bilinear  Time  Series  Models,  Stationarity ,  Ergodicity, 
Method  of  moments  and  Method  of  Least  Squares. 


1.  Introduction.  Let  X^,  t  e  Z  and  e^,  t  s  Z  be  two  stochastic 
processes  defined  on  some  probability  space  (fi,8,P)»  where 
Z  »  { .. .-1,0,1,. • .} .  X^,  t  e  Z  is  said  to  be  a  Bilinear  Model  with 

respect  ot  the  process  e^,  t  e  Z  if 

■  k  *  k  ^  k  k 


+  e^  a.e.[P]  for  every  t  in  Z. 


(1.1) 


We  designate  e^,  t  c  Z  to  be  the  input  or  unobservable  process  and 
X^,  t  e  Z  the  output  or  observable  process.  The  first  part  on  the 
right  side  of  (1.1)  can  be  Identified  as  the  auto-regressive  part  of 
the  process  x^,  t  e  Z,  the  second  part  as  the  moving  average  part  of 
X^,  t  e  Z  and  the  third  part  as  the  'pure'  bilinear  part  of  X^,  t  e  Z. 

A  study  of  bilinear  models,  therefore,  subsumes  the  study  of  auto¬ 
regressive, moving-average  and  mixed  auto-regresslve-movlng-average  models. 
There  are  two  basic  questions  that  arise  in  this  context. 

(1)  Suppose  e^,  t  e  Z;  a^'s;  b^'s  and  By ’a  are  all  given. 

Under  what  conditions,  is  there  a  process  X^,  t  e  Z  satisfying  (1.1)? 

We  assume  that  e^,  t  e  Z  is  Independent,  identically  distributed  with 
2  2 

E  e^  ■  0  and  E  e^  ■  a  <  «» • 

(2)  If  a  process  X^.,  t  e  Z  exists  satisfying  (1.1),  is  it 
strictly  stationary  and  also  ergodic? 


The  study  of  bilinear  models  was  initiated  by  Granger  and 
Andersen  [2]  and  Subba  Rao  [4].  The  problem  of  existence  of  a 
process  X^,  t  e  Z  conforming  to  (1.1)  has  been  studied  by  several 
authors  in  some  special  cases.  Tuan  Dinh  Pham  and  Lanh  Tat  Tran  [7] 
established  the  existence  of  a  strictly  stationary  process  X^.,  t  e  Z 


satisfying 


“  Sj.  +  a  +  b  a.e.[P] 

2 

for  every  t  e  Z  under  some  conditions  involving  a,b  and  a  .  Guegan  [3] 
studied  the  existence  problem  of  a  process  X^,  t  e  Z  satisfying 

“  ^t  +  b  X^_2  ®t-l 

for  every  t  in  Z.  Subba  Rao  and  Gabr  [6]  gave  a  set  of  sufficient 

conditions  for  the  existence  of  a  second  order  stationary  process 

Xj.,  t  e  Z  satisfying 

Vj  Vi  — 

for  every  t  in  Z.  Bhaskara  Rao,  Subba  Rao  and  Walker  [1]  showed  that 

under  the  same  set  of  sufficient  conditions  given  by  Subba  Rao  and 

Gabr  [6],  there  exists  a  strictly  stationary  process  X^,  t  e  Z 
satisfying 

■  'J  't-3 

for  every  t  in  Z.  This  model  is  more  general  than  the  one  considered 
by  Subba  Rao  and  Gabr  [6]  in  the  sense  that  p  and  q  given  above  could 
be  different.  All  these  models  are  special  casAs  of  (1.1).  The 
initial  step  in  the  proof  of  existence  theorems  given  in  Subba  Rao  and 
Gabr  [6]  and  in  Bhaskara  Rao,  Subba  Rao  and  Walker  [1]  is  to  rewrite 
the  above  models  in  vectorial  form.  We  pursue  the  same  line  of  tack  in 
studying  the  model  (1.1). 

In  all  the  bilinear  models  studied  in  the  literature,  the  moving 
average  part  has  not  been  included.  The  purpose  of  this  paper  is  to 
Include  the  moving  average  part  and  also  meet  the  following  goals, 
using  the  method  glnen  in  Bhaskara  Rao,  Subba  Rao  and  Walker  [1]. 


3 


(1)  Give  a  set  of  sufficient  conditions  under  which  there  is  a 

strictly  stationary  process  X^,  t  £  Z  satisfying  (1.1).  ((1.1)  has 

moving  average  component.) 

(2)  Show  that  such  a  process  is  also  ergodic. 

(3)  Show  that  such  a  process  is  unique 

(4)  Exploiting  the  ergodiclty  of  the  process,  obtain  estimators 

of  the  parameters  of  the  process  in  a  model  fitting  problem. 


Vectorial  representation 


We  represent  the  model  (1.1)  in  vector  form.  Suppose  the 
processes  X^,  t  e  Z  and  e^,  t  e  Z  satisfy  (1.1).  Let 

p  =  max  {r,m}, 
g  =  min  {m,4,} , 


q  *  max  {h,g}. 


(p  -  r) 


^1  ^2  ®  0  ...  0  0 


1  0  0  ...  0  0  0  ...  0  0 


0  1  0  ...  000  00 


0  0  0...0  0  0...  1  0 


(l>0>0j . . . >0) I 


(bj*0.0,...,0)»  j  -  1,2 . h, 


^  for  all  j  »h+l,  h+2,...,q 
when  h  <  g , 


0  0...0 

jj  j+l.j  mj 

0  0  ...0  0  0...0 


0  ...  0  0  0  0  , 


for  j  -  1,2 . . 


(p  -  m  +  j  -  1) 


■  0  for  j  “g+1,  g+2,...,q  when  h  >  g 


(X.^ ,  X.,  .,...,x.  _.,),  f  ^  Z, 


i*t  »* 


where  ^  is  the  matrix  of  appropriate  order  in  which  every  entry  is 
zero  and  T  stands  for  operation  transpose  of  a  matrix. 


Theorem  1  If  X^,  t  e  2  and  e^,  t  e  Z  satisfy  (1.1),  then 

t  — 1-1  — j  t-j 


X.  =  C  e. 


(2.1) 


for  every  t  in  Z. 

Proof.  By  direct  verification. 

We  restate,  in  view  of  the  above  theorem,  the  existence  problem 
as  follows. 

Let  e^,  t  e  Z  be  a  sequence  of  independent  identically  distributed 

2  2 

random  variables  with  E  e  ■*  0  and  E  e  ®  cr  <  “ .  Let  C  ,  A  ,  ^ , 

P’^P  pS 

j  *  l,2,...,q  and  ,  j  «  l,2,..,,q  are  given  matrices  with  real 
entries.  Is  there  a  strictly  stationary  process  X  ,  t  e  Z  satisfying 
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3.  Existence  theorem 

In  this  section,  we  show  that  under  the  condition  stipulated  in 
the  main  theorem  of  Bhaskara  Rao,  Subba  Rao  and  Walker  [1,  p.  106], 
the  strictly  stationary  process  X^,  t  e  Z  satisfying  (2.1)  not  only 
exists  but  also  ergodic  and  essentially  unique. 

Theorem  2  Let  e^,  t  e  Z  be  a  sequence  of  Independent  identically 

distributed  random  variables  defined  on  a  probability  space  (n,B,P) 

2  2 

with  E  e^  »  0  and  E  e^  =  a  <  ».  Let  A,  B,  ,  B.,,...,B  be  q  +  1 
t  t  1  /  q 

matrices  of  each  of  order  of  px  p  and 

-  A  ©  A  +  o^(Bj  ©  Bj^)  , 

Tj  -  o^(Bj  ©  (A^'^  Bj^  +  A^“^  B^  +. ..+  A  B^_j^) 

+  (A^‘^  Bj^  +  A^“^  B2  +...+  A  Bj_j^)  ©  + 

j  •  2,3, . . • ,q, 

where  ©  is  the  symbol  for  Kronecker  product  of  matrices. 

Suppose  all  the  eigenvalues  of  the  matrix 


F,  Fa...F  ,  F 
12  q-1  q 


2  2 
p  q  X  p  q 


I  0  ...  0  0 


0  I  ,...  0  0 

0  0  ...  1  ,  0 


have  moduli  less  than  unity.  Let  C  ,  b . ,  j  ■  l,2,...q  be  given  column 

pxl 

vectors  then  there  exists  a  strictly  stationary  and  ergodic  process 
t  e  Z  conforming  to  the  model 

P’'! 

■St  -  <= ‘t  +  *  2t-i  *  jij  't-J  *  j/j  St-J  V) 

for  every  t  e  Z.  Further,  if  a  process  U^,  t  e  Z  conforms  to  the  above 


bilinear  model  (3.1),  then 


a.e. [P] 


for  every  t  in  Z. 


Proof.  Let  the  process  ^  n,  t  e  Z  be  defined  as  follows. 


S  ^  =  0  if  n  <  0, 
-n,t 


C  e^.  if  n  -  0, 


^  h  n-j  ^  ‘t-i>  Vi.t-i 


^  o  7  7  •  • 

^  — n-2,t-z  t-2 


+BS  ^  e^  ,ifn>0 
q  -n-q,t-q  t-q 


for  every  t  in  Z.  We,  first,  observe  that  for  each  fixed  n, 

S  ,  t  e  Z  is  a  strictly  stationary  process.  Further,  for  each  fixed 
f  c 

n,  S  is  measurable  with  respect  to  the  a-field  o{e  ,  e 

for  every  t  in  Z,  where  o{e  ,  e  is  the  smallest  sub  a-field 

C 

of  B  with  respect  to  which  each  of  e^,  measurable. 

Consequently,  the  tall  a -field  of  (S  t  e  Z}> 

LJ  ^  1«***^CI  LJ  !.•••}  “  tail  a-field  of 

-n,t  -n,t-l  ^  t  t-1 

{e.,  t  e  Z}.  Hence  the  process  S  t  e  Z  is  ergodlc.  We  show  that 

C  il,  t 

lim  s  exists  almost  surely  [P]  for  each  fixed  t  in  Z.  If  X. ,  t  e  Z  is 
t  t 

»*• 

the  almost  sure  limit  of  S  n  ^  1  for  every  t  in  Z,  then  it  is 

c 


obvious  that  the  process  }L>  t  e  Z  conforms  to  the  bilinear  model  (3.1) < 


To  show  that  S  converges  almost  surely  as  n  -*■  »  for  every  fixed 
-n,  t 

t  in  Z,  define 


,t  *  -n,t  ’  ^-l,t  »  «»  t  e  Z. 

■  (A  +  Bj^  ®t-l^  •^-l  t-1  -n-2  t-2  ®t-2 


+. . .+  B  8  ^  e^ 

q  -n-q,t-q  t-q 


If  we  show  that  E  |  (s  ^)-|  i  it  follows  that  S  ,  n  ^  1 

converges  a.e.fP]  for  every  t  e  Z.  See  Step  2°  in  Bhaskara  Rao, 

Subba  Rao  and  Walker  [1,  p.  106],  where  X  Is  the  maximum  absolute 

eigenvalue  of  L,  (s  ) .  is  the  i^^  -  component  of  s  and  k  is 

c  1  “  » t 

some  positive  constant.  Following  Steps  3“  to  lO**  in  Bhaskara  Rao, 
Subba  Rao  and  Walker  [1],  one  can  show  that  E  I (s  ^).I  <  k  x”^^  for 
every  n  and  1  »  1,2, ...,0. 

The  limit  process  X^,  t  e  Z  is  also  ergodlc.  This  follows  from 
the  fact  that  is  measurable  with  respect  to  o{e^,  e^_j^,...}  for 
every  t  in  Z. 

Now,  we  prove  uniqueness.  Let  U^,  t  e  Z  be  a  process  conforming 
to  (3.1).  For  each  n  ^  q  and  t  e  Z,  let 


V  ■  U  -  S 


Then  -  (A  +  +  B^  V^.2,t-2 


+...+  B  V  ^  . 

q  -ti-q,t-q 


As  above,  ohe  can  show  that  E  ( (V  ,.}{{<  x"  for  every  n,  t  and  i. 

f  C  X 

Since  X  <  1,  it  follows  that  11m  V  0  a.e.[F].  But 

- 


(2)  The  most  Important  feature  that  emerges  by  comparing 
Theorem  2  above  and  the  main  theorem  in  Section  4  of  Bhaskara  Rao, 
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Subba  Rao  and  Walker  [1,  p.  106]  is  that  the  presence  of  moving  average 

part  makes  no  impact  on  the  existence  problem.  This  is  also  typical  of 

Linear  processes  as  the  following  corollary  shows. 

Corollary  3  Let  e^,  t  e  Z  be  a  sequence  of  Independent  identically 

distributed  real  random  variables  with  common  mean  0  and  variance 
2 

a  <  *.  Then  there  exists  a  strictly  stationary  process  X^,  t  e  Z 
satisfying 


e.[P] 


(3.2) 


for  every  t  in  Z  if  the  roots  of  the  polynomial 
f(x)  »  1  -  aj^x  -  a2X^  -  ...  -  a^x’^ 
are  in  absolute  value  greater  then  unity. 

Proof .  The  model  (3.2)  can  be  put  in  the  vector  form  as  follows. 
Let  p  ■  r,  q  •  i, 

ai  ®2  ®3  •••  ap_i 

1  0  0  ...  0 

A 

pxp  0  1  0  ...  0 

•  •  •  • 

«  •  •  •  •  •  • 

•  •  •  • 

0  0  0  ...  1 


m 

. J  “ 

Ixp 

m 

(1,0,0, .. .0) ,  and 

ixp 

Ixp 

m 

. \-p+l 

for  every  t  in  Z. 


(3.3) 


A  sufflclenc  condition  for  the  existence  of  a  strictly  stationary 
real  vector-vlaued  process  X^,  t  e  Z  satisfying  (3.3)  is  that  the 
maximum  absolute  eigenvalue  of  A  (B  A  is  less  than  unity.  This  con¬ 
dition  is  equivalent  to  the  condition  that  the  maximum  absolute  eigen¬ 
value  of  A  is  less  than  unity.  This  in  turn  is  equivalent  to  the 
condition  that  the  roots  of  the  characteristic  polynomial  are  less  than 
unity  in  modulus.  This  completes  the  proof. 


4.  On  estimation  of  the  parameters  of  a  bilinear  model 

Suppose  X^,  t  e  Z  is  a  real  valued  stochastic  process  satisfying 


’‘t  ■  n  +  i,  s-4  *  “ 


c-l  ^  “4  *c-4  't-4 


a.e. [P] 


for  every  t  in  Z  for  some  sequence  e^,  t  e  Z  of  Independent  identically 
distributed  random  variables  with  E  e^  *  0  and  E  e™  <  «  for  sufficiently 
large  m.  There  are  q  +  r  +  1  parameters  a,  a^^,  a^, . . .  ta^f  b^,  b2,...,b^ 
in  the  above  model  which  we  want  to  estimate  based  on  realizations 
Xj^,  of  X2,...,Xjj  respectively.  Assuming  the  process 

X^,  t  e  Z  to  be  strictly  stationary  and  ergodic,  we  proceed  with  the 
problem  of  estimation  as  follows.  (Assume,  for  simplicity,  that  the 
process  e^,  t  e  Z  is  completely  specified.) 

Step  1  Calculate  E  X^,  E  X^,...,  E 

A 

Let  E  X^  •  f ^ (a ,  a^ »  •  •  •  •  ♦ 

s  ■  l,2,...,q  +  r  +  l. 

Step  3  Estimates  based  on  the  Method  of  Moments  . 

Solve  the  equations 

fg(a,  Sj^,  a2,...,a^,  bj^,  b2,...,b^)  -  x^, 

s  ■  l,2,...,q  +  r  +  l 

AAA  AAA  A 

in  the  unknowns  a,  Sj^,  32*.. ••a^,  bj^,  b2»...»b^. 

This  method  is  justified  on  the  grounds  that  the  process 
X*,  t  e  Z  is  strictly  stationary  and  ergodic  for  s  •  1,2,3,...  and  that 

lim  i  ^  X®  -  E  X®  a.e.(P] 


for  every  s  ^  1. 

An  example 

Let  e^,  t  e  Z  be  a  sequence  of  independent  identically  distributed 
random  variables  with  e^  having  normal  distribution  with  mean  0  and 
variance  unity.  Let  X^,  t  t  Z  be  the  strictly  stationary  and  ergodic 


process  satisfying 


-  Cj.  +  a.Xj._^  +  b  X^_^  a.e.[P] 


for  every  t  In  Z  with  a  »  b  “  0.4.  It  can  be  checked  that  E  X 
and  E  X^  =■  (1  /  1  -a^  -  b^)  (1  +  2  b^(l  +  a)  /  (1-a)). 


t  1-a 


We  have  generated  1000  observations  x. ,  x^,...,x  from  the  above 

12’’  1000 

A  A 

model.  The  estimates  a  and  b  of  a  and  b  respectively  based  on  the 
method  of  moments  are  obtained  by  solving  the  equations 


a  1 

13  “lOOO  *1  ‘  °' 


5854906 


1  *2  "2 
1-a  -b 


x?.2. 


197616 


In  the  unknowns  a  and  b.  The  solution  Is  given  by  a  *  0.3963 

and  b  =  0.4281. 

Following  Subba  Rao  [4], [5],  one  can  apply  the  method  of  least  squares 

2 

to  minimize  Q(a,b)  *  I  over  a,b,  where  e^.  ■  ~  a  -  b  e^ 

t  «  1,2, ...  4-000,  to  obtain  the  following  recursive  equations. 

3®t 

SZ  *  *>  »t-i  —ZZ - *t-i' 


3b  *t-l  3b 


-'t-i  't-r 


T  *  ”  Vi 


-2  *  b  Vl 


■  ■  ='n-i 

.,2  ^  *t-l  3b 

3b 


,2  ,  ,2 

3  ®t  ^®t-l  ®  ®t-l 

3a3b  *t-l  sir  ^  *t-l  3a3b  " 


[iiiri] 
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,  .  ,  3Q(a.b)  3Q(a.b)  ^ 

Let  G  (a,b)  *  (  ■  ■■  ,  ^  g-g  --  )  and 


H(a,b) 


3^Q(a.b) 

3a2 


3  Q(a.b) 
3a3b 


3  Q(a.b) 
3adb 


3  Q(a..b) 
3b^ 


where  one  can  easily  check  that 


3Q(a.b) 

da 


3e. 


1000 

2  I  «t^‘ 

t»l  ^ 


3Q(a.b) 

3b 


3e. 


3^Q(a»b) 

d^a 


1000 

'  il  ''  "^b  • 
2 


1000 

2  I 

t-1 


3e 


t 

3a 


1000 

+  2  ^  e. 


3^ 


t-1 


3a 


t 

2  ’ 


3^b 


1000 

2  I 

t-1 


3e. 


13^  i 


aV 


1000  * 

+  2  I  e,  — 4 

t-1 


^  3b^ 


and 


3  Q(a.b) 
3a3b 


1000 
2  I 

t-1 


3e, 


de 


3a 


_0 

3b 


3a 


'3®t' 

’''*t 

.  3a  , 

3b 

b^°> 

< 

3b^ 

1 

1 


1000 

+  2  I  ®t 

t-1 


3^. 


_ t 

3a3b 


3^, 


3adb 


0,  we  obtain  the  Newton- 


Raphson  iterative  equation 


a 

b< 


(k+1)  ’ 

■  s^y 

(k+1) 

■ 

-  [H(a^*^\ 
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Starting  with  -  0.1,  using  the  recursive  equations  above, 

the  Newton-Raphson  method  gives 

a  -  0.3874 
b  -  0.3958 

correct  to  4  decimal  places.  These  values  are  very  close  to  the  true 
parameter  values  a  *  b  *  0.4.  Estimates  obtained  by  the  method  of 
moments  do  not  come  as  close  to  the  true  parameter  values  as  those 

obtained  by  the  method  of  least  squares  for  smaller  samples.  But  the 
method  of  moments  has  computational  simplicity  that  the  method  of  least 
squares  lacks.  One  could  use  the  estimates  given  by  the  method  of 
moments  as  the  starting-up  values  for  the  method  of  least  squares 
cranking  up  the  Newton-Raphson  machinery  in  small  samples.  However, 
if  the  sample  size  Is  large,  both  methods  give  values  very  close  to 
the  true  parameter  values. 

Concluding  Remarks 

We  have  shown  that  under  some  simple  condition  in  the  spectral 
radius  of  a  matrix.  Bilinear  models  do  exist,  are  stationary  and 
ergodlc.  Ergodicity  of  the  process  makes  the  method  of  moments  as  a 
natural  technique  for  adoption  to  estimate  the  parameters  of  the  model 
This  method  is  compared  with  the  usual  method  of  least  squares  and 
found  to  be  satisfactory  in  large  samples. 
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